Skip to main content

Survey Crawls

Survey crawls are run about twice a year, on average, and attempt to capture the content of the front page of every web host ever seen by the Internet Archive since 1996.



rss RSS

100,903
RESULTS


Show sorted alphabetically

Show sorted alphabetically

SHOW DETAILS
up-solid down-solid
eye
Title
Date Archived
Creator
collection

eye 2.5B

The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
Survey Crawl Number 8
collection
8,758
ITEMS
1.7B
VIEWS
collection

eye 1.7B

collection

eye 1.6B

The seeds for this crawl came from: 251 million Domains that had at least one link from a different domain in the Wayback Machine, across all time ~ 300 million Domains that we had in the Wayback, across all time 55,945,067 Domains from https://archive.org/details/wide00016 This crawl was run with a Heritrix setting of "maxHops=0" (URLs including their embeds) The WARC files associated with this crawl are not currently available to the general public.
collection

eye 1.7B

The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
collection

eye 1.3B

The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
Survey Crawl Number 7
Survey Crawl Number 7
collection
6,606
ITEMS
1.1B
VIEWS
collection

eye 1.1B

This "Survey" crawl was started on Feb. 24, 2018. This crawl was run with a Heritrix setting of "maxHops=0" (URLs including their embeds) Survey 7 is based on a seed list of 339,249,218 URLs which is all the URLs in the Wayback Machine that we saw a 200 response code from in 2017 based on a query we ran on Feb. 1st, 2018.   The WARC files associated with this crawl are not currently available to the general public.
collection

eye 940.4M

The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
collection

eye 811M

The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
collection

eye 478.5M

The seed for this crawl was a list of every host in the Wayback Machine This crawl was run at a level 1 (URLs including their embeds, plus the URLs of all outbound links including their embeds) The WARC files associated with this crawl are not currently available to the general public.
.com survey started January 2011
.com survey started January 2011
collection
2,535
ITEMS
630.5M
VIEWS
collection

eye 630.5M

Survey crawl of .com domains started January 2011.
Topic: webcrawl
Survey Crawl Number 9
collection
561
ITEMS
289.8M
VIEWS
collection

eye 289.8M

COM Survey Crawl 2009-2010
COM Survey Crawl 2009-2010
collection
729
ITEMS
96.6M
VIEWS
collection

eye 96.6M

COM survey crawl data collected by Internet Archive in 2009-2010. This data is currently not publicly accessible.
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl835.us.archive.org:survey from Fri Jun 9 20:38:43 PDT 2017 to Sat Jun 10 02:19:04 PDT 2017.
Topic: crawldata
survey_net00000
survey_net00000
collection
300
ITEMS
78.8M
VIEWS
collection

eye 78.8M

Survey crawl of .net domains started December 2010.
Topic: webcrawl
ORG Survey Crawls
ORG Survey Crawls
collection
193
ITEMS
41.5M
VIEWS
collection

eye 41.5M

Survey of .org domains. This data is currently not publicly accessible.
survey_net00001
collection
170
ITEMS
28.1M
VIEWS
collection

eye 28.1M

Survey crawl of .net domains started October 2011.
Topics: webwidecrawl, net
Survey Crawl Number 7
web

eye 215,571

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl844.us.archive.org:survey from Sat Jun 2 00:50:27 PDT 2018 to Fri Jun 1 23:10:33 PDT 2018.
Topic: crawldata
Survey Crawl Number 8
web

eye 1.3M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Thu Jan 24 00:44:19 PST 2019 to Wed Jan 23 18:36:19 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 1.2M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Thu Nov 29 20:01:41 PST 2018 to Thu Nov 29 17:30:37 PST 2018.
Topic: crawldata
Survey Crawl Number 7
web

eye 603,983

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl844.us.archive.org:survey from Sun Jul 1 12:35:48 PDT 2018 to Sat Jul 7 16:27:01 PDT 2018.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Sat Oct 14 11:17:53 PDT 2017 to Sat Oct 14 04:45:39 PDT 2017.
Topic: crawldata
Survey Crawl Number 8
web

eye 1.2M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl825.us.archive.org:survey from Mon Nov 26 06:50:00 PST 2018 to Mon Nov 26 06:16:37 PST 2018.
Topic: crawldata
Survey Crawl Number 8
web

eye 1.2M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Sat Jan 26 10:01:13 PST 2019 to Sat Jan 26 05:36:23 PST 2019.
Topic: crawldata
Survey Crawl Number 7
web

eye 608,252

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl842.us.archive.org:survey from Tue Jul 10 18:41:37 PDT 2018 to Tue Jul 10 15:02:56 PDT 2018.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Wed Oct 26 19:56:48 PDT 2016 to Wed Oct 26 15:40:51 PDT 2016.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Mon Oct 16 01:15:40 PDT 2017 to Sun Oct 15 18:45:40 PDT 2017.
Topic: crawldata
Survey Crawl Number 8
web

eye 946,125

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl838.us.archive.org:survey from Tue Jan 29 13:07:02 PST 2019 to Tue Jan 29 09:03:20 PST 2019.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Sat Oct 14 11:26:47 PDT 2017 to Sat Oct 14 04:54:28 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl805.us.archive.org:survey from Fri Feb 19 11:46:14 PST 2016 to Fri Feb 19 12:08:16 PST 2016.
Topic: crawldata
Survey Crawl Number 8
web

eye 840,897

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Sat Feb 2 11:45:54 PST 2019 to Sat Feb 2 07:05:21 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 867,012

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Sat Jan 26 18:17:50 PST 2019 to Sat Jan 26 14:00:14 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 874,084

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Thu Feb 28 04:49:14 PST 2019 to Wed Feb 27 23:44:56 PST 2019.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl413.us.archive.org:survey from Tue Jun 3 04:58:57 PDT 2014 to Tue Jun 3 01:09:27 PDT 2014.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl808.us.archive.org:survey from Sat Dec 27 23:31:46 PST 2014 to Sat Dec 27 21:07:24 PST 2014.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 951,100

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Wed Oct 11 23:22:24 PDT 2017 to Wed Oct 11 16:29:56 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl455.us.archive.org:survey from Sun Nov 30 05:27:12 PST 2014 to Sat Nov 29 22:49:06 PST 2014.
Topic: crawldata
Survey Crawl Number 8
web

eye 1.4M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Thu Jan 10 20:56:16 PST 2019 to Thu Jan 10 18:51:35 PST 2019.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl454.us.archive.org:survey from Wed Dec 11 08:04:19 PST 2013 to Wed Dec 11 01:03:53 PST 2013.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl337.us.archive.org:survey from Wed Dec 11 06:33:29 PST 2013 to Wed Dec 11 00:02:10 PST 2013.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Sun Jun 4 15:20:38 PDT 2017 to Sun Jun 11 13:26:56 PDT 2017.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl802.us.archive.org:survey from Tue Jan 20 16:21:48 PST 2015 to Tue Jan 20 08:54:08 PST 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl429.us.archive.org:survey from Tue Jun 3 14:36:58 PDT 2014 to Tue Jun 3 09:53:11 PDT 2014.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl429.us.archive.org:survey from Thu Dec 25 10:57:46 PST 2014 to Thu Dec 25 05:19:18 PST 2014.
Topic: crawldata
Survey Crawl Number 7
web

eye 190,192

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl849.us.archive.org:survey from Sat Jun 16 07:06:02 PDT 2018 to Sat Jun 16 04:12:21 PDT 2018.
Topic: crawldata
Survey Crawl Number 7
web

eye 223,277

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl848.us.archive.org:survey from Tue Apr 24 17:03:51 PDT 2018 to Tue Apr 24 17:48:22 PDT 2018.
Topic: crawldata
Survey Crawl Number 6: Sep 11th, 2017 - running now
web

eye 865,275

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Mon Nov 13 05:34:42 PST 2017 to Mon Nov 13 02:59:44 PST 2017.
Topic: crawldata
Survey Crawl Number 8
web

eye 125,170

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Thu Feb 28 06:54:49 PST 2019 to Thu Feb 28 04:57:29 PST 2019.
Topic: crawldata
Survey Crawl Number 7
web

eye 292,375

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl843.us.archive.org:survey from Thu Mar 8 11:40:32 PST 2018 to Thu Mar 8 06:55:08 PST 2018.
Topic: crawldata
Survey Crawl Number 7
web

eye 199,731

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl849.us.archive.org:survey from Wed Jun 13 05:49:57 PDT 2018 to Wed Jun 13 00:39:47 PDT 2018.
Topic: crawldata
Survey Crawl Number 7
web

eye 277,662

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl849.us.archive.org:survey from Fri Jun 22 07:23:21 PDT 2018 to Fri Jun 22 09:13:16 PDT 2018.
Topic: crawldata
Survey Crawl Number 8
web

eye 780,075

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl824.us.archive.org:survey from Sun Feb 3 10:15:14 PST 2019 to Sun Feb 3 09:18:16 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 329,601

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl344.us.archive.org:survey from Tue Jan 29 19:26:19 PST 2019 to Tue Jan 29 14:44:25 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 610,818

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Wed Jan 9 03:54:05 PST 2019 to Tue Jan 8 21:20:57 PST 2019.
Topic: crawldata
Survey Crawl Number 7
web

eye 1.1M

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl849.us.archive.org:survey from Wed Mar 7 16:06:50 PST 2018 to Wed Mar 7 10:55:57 PST 2018.
Topic: crawldata
Survey Crawl Number 8
web

eye 684,257

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Sun Feb 24 19:53:43 PST 2019 to Sun Feb 24 17:42:40 PST 2019.
Topic: crawldata
Survey Crawl Number 8
web

eye 498,454

favorite 0

comment 0

Internet Archive crawldata from Survey Webwide Crawl, captured by crawl818.us.archive.org:survey from Sat Dec 15 09:02:19 PST 2018 to Sat Dec 15 09:38:26 PST 2018.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl808.us.archive.org:survey from Sat Aug 1 00:45:26 PDT 2015 to Sat Aug 1 11:47:09 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl800.us.archive.org:survey from Sat Aug 1 00:47:24 PDT 2015 to Sat Aug 1 11:53:48 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl803.us.archive.org:survey from Sat Aug 1 00:46:17 PDT 2015 to Sat Aug 1 12:00:07 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl805.us.archive.org:survey from Sat Aug 1 00:46:27 PDT 2015 to Sat Aug 1 12:03:29 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl836.us.archive.org:survey from Sat Aug 1 00:43:43 PDT 2015 to Sat Aug 1 11:44:39 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl815.us.archive.org:survey from Sat Aug 1 00:44:26 PDT 2015 to Sat Aug 1 11:53:01 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl807.us.archive.org:survey from Sat Aug 1 00:45:46 PDT 2015 to Sat Aug 1 11:59:41 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl423.us.archive.org:survey from Sat Aug 1 00:49:35 PDT 2015 to Sat Aug 1 06:06:54 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl813.us.archive.org:survey from Sat Aug 1 00:44:39 PDT 2015 to Sat Aug 1 11:49:23 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl811.us.archive.org:survey from Sat Aug 1 00:44:50 PDT 2015 to Sat Aug 1 11:44:35 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl809.us.archive.org:survey from Sat Aug 1 00:45:20 PDT 2015 to Sat Aug 1 11:48:26 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl817.us.archive.org:survey from Sat Aug 1 00:38:37 PDT 2015 to Sat Aug 1 11:36:18 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl812.us.archive.org:survey from Sat Aug 1 00:44:56 PDT 2015 to Sat Aug 1 11:40:06 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl837.us.archive.org:survey from Sat Aug 1 00:43:49 PDT 2015 to Sat Aug 1 11:37:56 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl802.us.archive.org:survey from Sat Aug 1 00:46:09 PDT 2015 to Sat Aug 1 11:55:45 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl806.us.archive.org:survey from Sat Aug 1 00:46:36 PDT 2015 to Sat Aug 1 12:04:06 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl339.us.archive.org:survey from Sat Aug 1 00:38:24 PDT 2015 to Sat Aug 1 06:11:57 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl835.us.archive.org:survey from Sat Aug 1 00:43:36 PDT 2015 to Sat Aug 1 11:51:01 PDT 2015.
Topic: crawldata
Internet Archive crawldata from Survey Webwide Crawl, captured by crawl801.us.archive.org:survey from Sat Aug 1 00:47:17 PDT 2015 to Sat Aug 1 11:58:58 PDT 2015.
Topic: crawldata